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Abstract: Defining organizational objectives, ordering priorities in relation to costs and benefits, 
and measuring progress is difficult in higher education because of the complexities involved in 
evaluating program effectiveness. In spite of these difficulties, the problem of measuring 
performance must be solved in ways that can yield meaningful results for administrators, 
educators, and the various constituencies they serve. This study provides an overview of the 
process of assessing institutional effectiveness relative to student outcomes in an adult continuing 
education program. The results of an investigation of the effects of motivation and classroom 
environment on student satisfaction are presented. 
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Introduction 

"Continuing education" at the university level has evolved to embrace the notion that learning is 
a lifetime endeavor. In addition to serving degree-seeking students, noncredit continuing education 
classes are offered to meet ongoing professional and personal educational needs and are a 
significant part of university outreach efforts across the nation. 

At a time when institutions of higher education are faced wiih greater demands for 
accountability and dwindling resources to fund a myriad of educational programs, continuing 
education units are being called upon to produce tangible evidence that they are meeting significant 
needs. Continuing educators are also facing funding cutbacks and the administrators are being 
required to consolidate and prioritize programs. 

To some degree, performance can be quantified by such measurements as numbers of students 
served and programs provided. Though these are important indicators of performance, more 
adequate measures of program effectiveness relative to the quality and responsiveness of services 
are needed. Such additional measures are of crucial importance to educators if they are to meet the 
demands for greater accountability and effectively set objectives and priorities for the future. 

Review of Literature 

The need to measure program effectiveness in higher education has resulted in the development 
of a number of program performance measures and considerable interest in student outcomes such 
as achievement and satisfaction (Bean & Bradley, 19X6; Pike, 1991). In adult and continuing 
education, progress has been limited because of inherent difficulties in assessing nontraditional 
programs and students. Much of the research efforts thus far have been dominated by a search for 
factors that determine persistence (Aitken, 1982; Anderson & Darkenwald, 1979; Garrison, 1985). 
In terms of noncredit programs where students are not seeking an easily identifiable, long-term 
goal however, the concept of persistence has limited applicability. Given this limitation as well as 
a program in which there are no grades or academic credit, student satisfaction may be viewed as 
an especially appropriate criterion for measuring program effectiveness. 
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While measures of satisfaction vary a great deal in the models used and in the extent of 
developmental effort, there has been a sustained effort at evaluating student satisfaction in terms of 
classroom instruction (Urdang, 1982; see Rosenblum & Darken wald, 1983). Investigators have 
also conceptualized student satisfaction as being related to multiple domains. In addition to 
instruction, they have examined other aspects of satisfaction with the college experience such as 
institutional policies and procedures, physical conditions, the social environment, and institutional 
climate (American College Testing Program, 1992; Betz, Klingensmith & Menne, 1969; Carder., 
1978; Hendry, 1983; Mangano & Corrado, 1979; Pace, 1964 & 1979). 

Research on satisfaction in higher education has also focused on the congruency between 
student and the college environment, and a number of studies suggest that discrepancies in 
perceived self -college similarity are significantly related to dropout and dissatisfaction. (Holland, 
1973; Morstair, 1977; Nafziger, Holland, & Gottfredson, 1975; Pervin, 1967; Penan & Rubin, 
1987). Researchers such as Hearn (1985) have also found that the processes of satisfaction 
formation are net uniform across groupings of students by academic major and gender. 

Several researchers in higher education have hypothesized that the interactional process 
experienced by students within the classroom itself, has a profound effect on learning and 
achievement. (DeYoung, 1977; Ennis, et al., 1989; Menges & Kulieke, 1984). Their results 
suggest that students have clear expectations about their classes, and an experience in the 
classroom that is incongruent with expectations leads to dissatisfaction. 

Purpose 

The purpose of the study was to determine whether certain motivational and classroom 
environment factors have an effect on student satisfaction. The functional relationships between 
motivation, perceived classroom environment, and student satisfaction, were investigated for each 
of three major curricular groups: arts and leisure programs, personal development programs, and 
professional development programs. Besides the three curricular groups, additional variables that 
may affect satisfaction as an outcome were investigated. These included student characteristic 
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variables of gender, age, ethnicity, and level of education, and contextual field variables such as 
the size and length of class, and the experience of the teacher. 

Research Questions 

The first question of the study was designed to investigate whether it was possible to classify 
students into the three curricular groups, based on their characteristics, reasons for participation, 
and perceptions of their classroom environment. 

The second question concerned itself with differences in student satisfaction. Specifically, it 
investigated whether students differed significantly in their perceptions of satisfaction when 
grouped by curricular offering. 

Finally, the study sought to determine whether there was a set of variables that could explain 
significant amounts of variance in student perceptions of satisfaction. 

Methodology 

Subjects 

A stratified random sample of the population was studied. The population was stratified by 
course curricular group, length, and size. Data was collected in the classroom from students 
enrolled in noncredit courses at the College of Continuing Education and Community Service at the 
University of Hawaii. Survey responses were entered into a database which provided various 
administrative reports and a nummary of the satisfaction survey by class for ongoing program 
management. Responses from 1 180 students were subsequently utilized in the analysis. 

It should be noted that the noncredit program presented an unusual sampling problem. Unlike 
most collegiate programs, it consists of courses of varying lengths and dates. Although each 
noncredit term consists of 15 weeks, courses are scheduled throughout the term and do not begin 
and end uniformly. Additionally, registration for classes begins approximately three weeks prior 
to the start of the term and is continuous, with no specific enrollment period. Given the nature of 
the program and the fact that the survey instruments were administered in the classroom, it was 
necessary to utilize a computerized sampling frame which was based upon estimated enrollment. 
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ill 

This investigation sought to integrate constructs and evaluative structures developed in three 
streams of research. Motivational factors were evaluated by the Education Participation Scale 
(Boshier, 1991), a factor-analytic measure of motive for attendance which measures the extent to 
which students participate in order to: (1) improve communication skills (COM), (2) socialize with 
others (SOC), (3) obtain more education and training (EDUC), (4) advance or maintain their 
professional careers (ADV), (5) enhance the quality of their family life (FAM), (6) participate in 
meaningful and enjoyable activities (STIM), and (7) satisfy cognitive interests (COG). 

Classroom environment factors were measured by the Adult Classroom Environment Scale 
(Darken wald 1987) which measures environmental factors in the learning environment including: 
(1 ) the amount of student involvement in the learning situation (INVL), (2) the opportunities for 
affiliation with others (AFFL), (3) the level of teacher support (SUP), (4) the emphasis on task 
completion (TASK), (5) the orientation towards personal goals (PGOAL), (6) the organization and 
clarity of instruction (ORG), and (7) the amount of student influence (INFL). 

Student satisfaction was evaluated by the Dimensions of Adult Student Satisfaction (Hendry, 
19X3), which measured five dimensions of student satisfaction: (1) satisfaction with program 
quality; satisfaction with institutional climate as measured by the quality of relations with (2) staff, 
and (3) students; (4) satisfaction with support services; and (5) satisfaction with physical facilities. 

Principal components factor analysis was used to verify the stability of the factor structure of 
each instrument, and varimax orthogonal rotation was used to produce the final structure. Internal 
consistency reliability evaluations were conducted for all three instruments. The overall reliability 
(alpha) was .96 for the Dimensions of Adult Student Satisfaction (DASS), .92 for the Educational 
Participation Scale (EPS), and .90 for the Adult Classroom Environment Scale (ACES). 
Statistical Analysis 

Stepwise discriminant f unction analysis was utilized to determine whether students could be 
classified into the three curricular groups. The discriminant analysis procedure took the seven EPS 
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subscales, the seven ACES subscales, and the four student characteristic variables in combination 
to determine whether students could be classified into curricular groups. Analysis of variance was 
used to evaluate the null hypothesis relative to significant differences in dimensions of satisfaction 
among curricular groups, followed by appropriate multiple comparison tests. To investigate the 
question concerning the predictive validity of selected variables on student satisfaction, stepwise 
multiple regression analysis was utilized. 

Results 

Curricular Group Classification 

As reported in Table 1, the discriminant function analysis found that a significant separation 
was possible among the three curricular groups. The separation was based on the utilization of 
two functions which produced a significant difference beyond the p < .01 level. 

TABLE 1 



Canonical Discriminant Function: Three Group Solution (N = 1036) 



FUNCTION 


% OF VARIANCE 


CANONICAL 


WILKS' 


CHI- 


DF 


SIGNIF 




ACCOUNTED FOR 


CORRELATION 


LAMBDA 


SQ 






1 


72.8 


.6294 


.4850 


742.36 


30 


.00** 


2 


27.2 


.4437 


.8032 


224.90 


14 


.00** 



** p < .01 



The first canonical function accounted for 73% of the variance, and the second accounted for 
the remaining 27%. The canonical correlation, an indicator of a function's ability to discriminate 
among groups was .63 for Function 1, and .44 for Function 2. 

Each of the selected variables was entered in stepwise fashion to evaluate its potency in making 
the separation between groups. The stepwise procedure reported in Table 2 identified 15 variables 
as predictors, and rank ordered them in terms of importance in separating the groups. Motivation 
variables appeared to be the most potent discriminators in separating the groups. Significant 
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classroom environment variables which were entered included emphasis on task, orientation 
towards personal goals, level of teacher support, and the amount of student involvement and 
infiuence in the learning situation. All four student characteristic variables were identified as 
significant predictors and included student age, ethnicity, gender, and education. 

TABLE 2 

Summary of the Stepwise Discriminant Function Procedure Using EPS, ACES and Student 



Characteristic Variables to Define Curricular Groups (N = 1036) 



STEP 


VARIABLE (SCALE) 


WILKS' 


STGNTF 


STANDARD 


STANDARD 






LAMBDA 




CAN COEF 


CAN COEF 










FUNC 1 


FUNC 2 


1 


ADV (EPS) 


.81 


.00** 


-.67 


-.64 


2 


COM (EPS) 


.70 


.00** 


-.66 


.7S 


3 


SOC (EPS) 


.61 


.()()** 


.37 


-.32 


4 


FAM (EPS) 


.56 


.00** 


.19 


.53 


5 


TASK (ACES) 


.54 


.00** 


.22 


-.13 


6 


COG (EPS) 


.53 


.00** 


.15 


.21 


7 


STUDENT AGE 


.52 


.00** 


.24 


-.07 


8 


STIM (EPS) 


.51 


.00** 


.25 


-.11 


9 


PGOAL(ACES) 


.51 


.00** 


.27 


.08 


10 


SUP (ACES) 


.50 


.00** 


-.13 


-.25 


11 


ETHNICITY § 


.49 


.00** 


-.01 


.25 


12 


STUDENT GENDER 


.49 


.00** 


.14 


.06 


13 


INVL (ACES) 


.49 


.00** 


-.09 


.12 


14 


INFL (ACES) 


.49 


.(){)** 


-.06 


-.08 


15 


EDUCATION 


.49 


.(H)** 


-.03 


-.10 



8 Caucasian/Not Caucasian 
** p < .01 



The most accurate membership prediction was for the arts and leisure group (77%), 
followed by the professional development (69%), and the personal development (57%) groups. 
Differences in Satisfaction Among Curricular Groups 

A summary of the mean scores from the ANOV A and Scheffe for each of the subscales of the 
Dimensions of Adult Student Satisfaction by curricular group, is presented in Table 3. 
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TABLE 3 

A Summary of the Mean Scores from the ANOVA and Scheffe for Each of the Subscales 
of the Dimensions of Adult Student Satisfaction by Curricular Group 



CURRICULAR GROUP 



ARTS /LEISURE 
N=475 



PERSONAL PROFESSIONAL 
N=408 N=25l 



48.1 



DIMENSIONS OF ADULT STUDENT SATISFACTION 
Program Quality 
Staff Relations 
Student Relations 
Support Services 



46.2 



46.7 



39.7 



38.4 



39.2 



12.3 



11.4 



11.5 



24.3 



23.4 



23.4 



Physical Facilities 



30.1 



28.7 



28.9 



NiH£. Boxed values indicate groups that were significantly different at the p < .05 level. 

The results of the analysis indicated that there were significant differences among curricular 
groups on all five of the satisfaction subscales. Although there were no significant differences 
between the personal and professional development groups on any of the subscales, the Scheffe test 
and subsequent examination of means revealed that with the exception of satisfaction with staff 
relations, the arts and leisure group had significantiy higher satisfaction scores than each of the 
other two groups. For the staff relations subscale, the arts group had significantly higher 
satisfaction scores than the personal development group, while there were no significant differences 
between the arts and professional groups. 
Predictors of Satisfaction 

Multiple Regression Analysis was used determine whether the selected set of variables 
(motivation factors, classroom environment factors, class size and length, and teacher experience), 
could explain significant amounts of variance in student satisfaction. For each of the three 
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curricular groups, five stepwise multiple regression procedures were conducted utilizing each of 
the satisfaction subscales. Table 4 presents the results of the analysis indicating significant 
predictors for the various dimensions of student satisfaction for each of the three curricular groups. 
The number of steps, required to produce the prediction equation, the predictor variables, /?, 
R-square, B values, and Beta weights are reported. Taken together, the first two variables in the 
equations developed for all groups accounted for most of the total variance of the predictors of 
satisfaction, with variables entered in subsequent steps accounting for 2% or less of the variance. 

In terms of student satisfaction with program quality, organization and clarity entered first for 
all curricular groups. Teacher support was entered at the second step for the arts and personal 
development groups, while affiliation was entered for the professional group. In terms >. 5 student 
satisfaction with staff relations, organization and clarity entered first for the arts and professional 
groups, while personal goal attainment entered first for the personal development group, Personal 
goal attainment entered at the second step for the arts group, teacher support entered for the 
personal development group, and affiliation entered for the professional group. 

Affiliation entered into the prediction equation at the first step for student satisfaction with 
student relations for all groups. Personal goal attainment entered at the second step for the arts 
group, while organization and clarity entered for both the personal and professional development 
groups. Although class length was a variable included in the prediction equations of all groups, an 
inverse relationship occurred only for the professional group. For this group, class length had the 
effect of lowering satisfaction scores while for the other groups it had the effect of raising scores. 

For all groups, organization and clarity was entered into the prediction equation at the first step 
for student satisfaction with support services. For the personal development group, personal goal 
attainment entered at the second step while for the professional group, affiliation entered second. 
No additional variables were included for these groups after the second step. For the arts group, 
communication improvement motive entered at the second step, followed by personal goal 
attainment. 
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In terms of student satisfaction with physical facilities, the three curricular groups differed in 
the variables entered at all steps of their respective prediction equations. Organization and clarity 
along with teacher experience entered in the first two steps for the arts group, task orientation and 
personal goal attainment entered for the personal development group (with no other variables 
entering for this group), and affiliation and organization and clarity entered for the professional 
group. Although class size was a variable included in the prediction equations of both the arts and 
professional development groups, an inverse relationship occurred for the professional group. For 
the arts group, larger class size had the effect of increasing satisfaction with physical facilities, 
while the inverse was true for the professional group. 

Discussion 

Introduction 

This study provided a laboratory for studying learners who voluntarily participate in the 
learning experience, and who are primarily motivated by the desire to satisfy the need to know, 
unaffected by grades or the requirements of a degree program. Although this study empirically 
validated Boshier's (1991) Education Participation Scale and Darkenwald's (1987) Adult 
Classroom Environment Scale with a relatively large sample of students from Hawaii, the 
emphasis of the investigation was on the application of this research to formulating a better 
understanding of student satisfaction. The analyses focused on the motivations students bring to 
the learning situation and on defining the situation itself by the various components of the 
classroom environment, and sought to explore the impact these variables may have on satisfaction. 
Student Characteristics and Motivation 

The first phase of this study developed a basis for classifying a student body that was 
participating in a program with a very broad curriculum. By utilizing the concepts developed in 
typological research on the adult learner, it advanced the findings of previous studies by first 
dividing students into curricular groups, and then specifying the demographic characteristics and 
motives for participation that defined these groups. It was assumed that this classification would 
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provide a means for better understanding the characteristics and motivations of a diverse student 
population, and thereby provide important typological information for program administrators. 

The findings revealed that each curricular group had a distinctive set of student characteristics 
and reasons for participation, and no single variable adequately described any one group. This is 
similar to findings reported by Morstain and Smart (1977), and confirms their conclusion that 
grouping adult students on the basis of demographic characteristics alone lacks precision in 
describing them. Motivations in combination with student characteristics were powerful 
discriminators among groups, and the results showed significant variation across different 
groupings of learners. 

In terms of motivation, the results validated Boshier's ( 1991) adult learner typology and the 
effectiveness of the Education Participation Scale in differentiating among a diverse group of 
students with varying reasons for participating in continuing education. The findings indicated that 
the Education Participation Scale was useful in defining the salient differences in the motivations for 
participation among defined curricular groups, and provided constructs for understanding student 
motivation that could be related to variation in satisfaction in subsequent analyses. The findings 
also provided some indication of the relative importance of these constructs or factors in 
discriminating among the three curricular groups. 
Classroom Environment 

Much of the research in the field has focused on the identification of the characteristics of 
students who participate in educational activities, and on the development of psychometrically 
sound instrumentation for measuring the reasons for their participation. Almost no research in 
adult education has focused on understanding the classroom environment and the environment's 
impact on outcomes such a!, satisfaction. Utilizing psychometrically sophisticated instruments, this 
study was successful in relating the interaction of the adult learner in a classroom environment to 
the outcome of satisfaction, and therefore contributes a unique perspective for educators and 
administrators in the field. 
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• Just as the Education Participation Scale was useful in defining the salient differences in 
motivations for participation among defined curricular groups, this study demonstrated the 
usefulness of the Adult Classroom Environment Scale in measuring variability Ln students' 
perceptions of their classroom environments among students in the three curricular groups. The 
results indicated that students in each curricular group had discriminating characteristics and 
patterns of motivation, and preferred particular kinds of classroom environments which 
emphasized a different mix of the environmental domains conceptualized by Moos (1976) and 
Darkenwald (1987). The findings also provided a conceptual framework for understanding the 
various dimensions of the classroom environment and some indication of the relative importance of 
these dimensions in discriminating among the three curricular groups. 
Student Satisfaction 

The results of this study confirmed Hendry's conceptualization of the multi-dimensional nature 
of satisfaction. Although curricular groups shared some similarities which might be expected, the 
analyses revealed differences among groups, as well as among the various dimensions of 
satisfaction. Thus, the findings give considerable theoretical support for viewing satisfaction as a 
complex phenomenon. The findings also suggested that the development of satisfaction is not 
uniform across curricular groups, and given the differences in characteristics, motivations, and 
classroom environments among the groups, it would appear that satisfaction is specific and 
complex, and therefore a simple uni-dimensional conception is inadequate. 

The results confirmed previous findings in the literature (Bean and Bradley, 1986; Pervin, 
1967; Pervin & Rubin, 1987) which suggested a significant relationship between student attributes 
and satisfaction Previous studies have indicated that learning environments may have differential 
impacts on students (Hearn, 1985; May, 1985; Morstain, 1977), but little research has been done 
which has linked specific dimensions of the classroom environment to specific indices of student 
satisfaction. For example, findings from this investigation suggested that classes that emphasized 
organization and supportive student/teacher relationships, had a positive impact on student 
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satisfaction with program quality. However, the results also indicated that some emphasis on all 
environmental domains facilitates positive outcomes and as suggested by Moos (1980), a single- • 
minded emphasis on only one domain may have negative effects. For instance, while organization 
and teacher support may be of key importance in the learning situation, class discussions and other 
such opportunities for students to interact with one another may also be important components in a 
satisfying learning experience. 

The findings also suggested that certain types of learning environments may be more satisfying 
to students with certain types of learning motivations, who seek to satisfy their needs in particular 
kinds of classes. For example, while emphasis on task may encourage cognitive growth for some 
students participating in personal development classes, it may have a negative impact on other 
students who may be participating in leisure activities for social purposes. 

The results validated the importance of understanding the classroom environment, and 
suggested that such an understanding could be a useful tool in helping teachers understand the field 
dynamics of their classrooms from the students' viewpoint. While teachers may have little control 
over who they teach and such variables as scheduling and logistical arrangements, they do have the 
opportunity to improve the environment of their classroom and in doing so, may significantly 
improve the learning experience of their students. The relative importance of the classroom 
environment on student satisfaction suggests that what the student experiences in the classroom may 
have such a profound influence on satisfaction that it may be possible to enhance satisfaction 
through purposeful practices designed to improve the quality of the classroom environment. 

While the findings of this study confirmed some of the assumptions commonly held by 
practitioners in the field, it also pointed towards ways that might improve the quality of the 
educational experience provided to students. Fundamentally, the findings revealed that there can 
be no substitute for providing adults with a learning experience that is well-organized and clearly 
delivered. The findings suggested that students value the opportunity to interact with others while 
learning, even in instances in which they may be participating for career and job-related purposes 
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rather than for social reasons. Additionally, the findings identified some of the more subtle 
components that may be essential ingredients for a satisfying learning experience, and may have 
the potential for assisting educators in making more discriminating decisions about the design and 
delivery of education programs. These include other components of the classroom environment 
such as the support and encouragement the teacher directs towards students, the emphasis placed 
on task and learning objectives, and the extent to which students can pursue individual goals, 
influence course content, and are actively involved in class activities. The findings also suggested 
that contextual or situational variables such as class length and size, and the experience of the 
teacher, may have a significant impact on the learning experience. 

This investigation shifted the analytical emphasis away from general student demographics 
and classification of motives to focus attention on what needs learners might hope to satisfy by 
participating in courses, how learners perceive their learning environments, and an indication of 
how environmental components in interaction effect the perception of value gained from 
participation. While it was limited to the measurement of satisfaction and was not able to assess 
the impact of other outcomes of participation, it was successful in relating perceived components of 
the environment to measures of student satisfaction with the learning experience, and in so doing, 
provides insights into how satisfaction occurs, and how students' perception of satisfaction is 
inhibited or enhanced within the classroom environment. 

Recommendations 

Many adult and continuing education programs are self-supporting and so the bottom line is 
that sufficient revenues from course fees must be generated to cover the expenses of providing the 
programs. However, the measurement of program effectiveness must always move beyond this 
basic criterion to the learner, and such considerations as his or her feelings of satisfaction with the 
learning experience. Our success as educators and administrators depends upon our ability to 
identify and meet the needs of a wide spectrum of learners. Programming that is driven solely by 
its ability to generate income overlooks important needs and the fundamental aesthetics of learning. 
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This study demonstrated that collecting data on students is within the reach of most 
organizations and even if carried out on a limited basis, can provide meaningful information on 
program effectiveness. Particularly for noncredit programs where there are few admission 
requirements and forms to fill out prior to attendance, information that may be collected on students 
is often overlooked. More work is needed to help administrators to not only identify the salient 
characteristics of their particular student population, and the relationship those characteristics may 
have to the classroom experience and satisfaction — but also in assisting them in developing feasible 
ways for collecting and managing data. 

Additional research might focus on defining dimensions of satisfaction that are appropriate to 
target populations. That is, additional work is needed on developing indices of satisfaction that 
differentiate among and are appropriate for, the broad spectrum of learners and learning 
environments in higher education. Additionally, the interrelationships among the various 
dimensions of satisfaction need exploration. So for instance, future studies might seek to 
determine whether dissatisfaction with support services has an adverse effect on satisfaction with 
program quality. Further, research on how satisfaction with an experience might affect future 
expectations and participation is needed to assess the impact of outcomes on future experiences. 

Finally, given the fact that at least for the present investigation, measurable constructs were 
related to an important and meaningful dependent variable, further studies with students in other 
environments and institutions are called for in order to extend and confirm these important findings. 
Implications for Practice 

Evaluating program effectiveness is difficult in higher education because of the complexities 
involved. However, educational institutions do formulate goals and objectives, and become deeply 
committed to them. They are also held accountable by Federal agencies, legislators, governors, 
state coordinating bodies, boards of trustees, and the public. Therefore, the problem of measuring 
performance must be solved in ways that can yield meaningful results for administrators, educators, 
and the various constituencies they serve. 
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While there is no one methodology or single criterion that will allow us to absolutely measure 
the effectiveness of the programs we provide, we can assess important outcomes like student 
satisfaction. We can use the results of these analyses to evaluate the strengths and weaknesses of 
our programs, and to guide us in effecting improvements that will be of benefit to the learners that 
we serve. If we are able to understand how we may have gotten to where we are today in 
satisfying the needs and aspirations of our students, then perhaps we will have the vision and 
inspiration for contemplating what we might yet become, and where we might go in the 
tomorrow's ahead of us. 
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